Embodiments of the inventive concept relate to a sampling method and image processing apparatus of RANSAC and CS-RANSAC algorithms for obtaining plane homography between two images.
In recent years, Augmented Reality (AR) systems are attracted as effective means for visualizing weather information which can be expressed in texts, still images, animations, videos, displays for mobile devices, or 3D objects of cameras. In some applications, AR systems are steadily studied to enable highly complex and microscopic works such as maintenance and repair of aircrafts.
To accurately synthesize virtual objects on camera images in an AR system, it is necessary to precisely estimate poses of the virtual objects. Those virtual object poses may be estimated by calculating homography matrixes between reference images and camera images.
The homography matrix is calculated through a Random Sample Consensus (RANSAC) algorithm and each sample set is selected at random. Hereat, the accuracy of the homography matrix generated by the RANSAC algorithm is considerably dependent on features which are randomly selected.
A general RANSAC algorithm usually operates to select features without regarding positional correlations between the features. During this, sets of the selected features form linearity or placed so closely, finally degrading the accuracy of the homography matrix.
FIGS. 1 and 2 illustrate pixels where reference images are mapped with camera images.
Referring to FIG. 1, pixels included in the reference images of the left are mapped to camera images of the right. This pixel mapping is carried out by means of a homography matrix which is calculated from selected four pairs of features. In FIG. 1, the feature pairs are indicated by yellow circles. As shown in FIG. 1, if the selected features form linearity, the features just can accurately estimate poses of objects which are placed in a selected specific area (e.g. the area in the yellow rectangular box of FIG. 1), but inaccurate in estimating poses of objects placed in other areas (e.g. a pitch trimmer indicated by the orange circuit). As shown in FIG. 2, even if selected features are placed in a distance, it is difficult to accurately estimate poses of objects as like the case of linearity. In other words, the accuracy of the homography matrix is degraded to be worse.
A general RANSAC algorithm is formed of hypothesis and estimation steps. In the hypothesis step, features are selected at random. In the estimation step, the consistency between two features (e.g. the number of inliers) is inspected. Those two steps are repeated until a process of finding better features for representing datasets becomes failed. As such, to raise the accuracy of homography matrix, there are proposed an LO-RS algorithm, a T-RS algorithm, an MF-RS algorithm, and so on.
However, since homography matrixes calculated by the LO-RS algorithm, the T-RS algorithm, and the MF-RS algorithm also include insignificant samples (features laid on linearity or crowded in a specific area), it is difficult to accurately estimate poses of objects throughout the whole area of image. That is, the accuracy of homography matrix is still standing in a low level.
Therefore, when estimating a plane homography matrix between two images in an AR system requiring a powerful chase function, there is a need of technology for accurately estimating the homography matrix to the whole area not to a specific area.